ASURA: Scalable and Uniform Data Distribution Algorithm for Storage Clusters

In algorithm management, data are distributed in accordance with a data distribution algorithm that is capable of determining, on the basis of the datum ID, the node in which the required data is being stored. Among the requirements for a data distribution algorithm are short calculation times, low memory consumption, uniform data distribution in accordance with the capacity of each node and the ability to handle the addition or removal of nodes. This paper presents a data distribution algorithm called ASURA (Advanced Scalable and Uniform storage by Random number Algorithm), which satisfies these requirements. It offers roughly 0.6-μs calculation time, kilobyte-order memory consumption, less than 1% maximum variability between nodes in data distribution, data distribution in accordance with the capacity of each node and optimal data movement to maintain data distribution in accordance with node capacity when nodes are added or removed. ASURA is contrasted in this paper qualitatively and quantitatively with representative data distribution algorithms: Consistent Hashing and Straw Buckets in CRUSH. The comparison results show that ASURA can achieve the same storage cluster capacity as Consistent Hashing with dozens fewer nodes by virtue of the uniformity of its distribution with the same level calculation time. They also show that the execution time of ASURA is shorter than that of Straw Buckets in CRUSH. The results reveal that ASURA is the best algorithm for large-scale storage cluster systems. rowth in size of data managed by computers is leading to increased storage system capacity. Recent capacities cannot be achieved with one storage node or a few storage nodes. Thus, storage cluster technologies and distributed storage technologies, which manage many storage nodes as one storage system, are urgently required. Each node in a storage system knows all the nodes in the storage cluster and each node in a storage system knows some of the nodes in a distributed storage system (peer-to-peer (P2P) system). Storage clusters are the focus of this paper. Because applications need to know data-node correspondences for the purpose of access, all combinations of data identifiers (IDs) and data-storing nodes must be managed. When data need to be accessed, the node storing them is determined on the basis of the relevant datum ID. There are three such types of combination management: table management, algorithm management and a mixture of both. In table management, combinations of data IDs and data-storing nodes are memorized in a management table. When a datum …